Assessment of Interjudge Reliability in the Open-Ended Questions Coding Process
نویسندگان
چکیده
منابع مشابه
Assessment of Interjudge Reliability in the Open-Ended Questions Coding Process
In the process of coding open-ended questions, the evaluation of interjudge reliability is a critical issue. In this paper, using real data, the behavior of three coefficients of reliability among coders, Cohen’s K, Krippendorff’s α and Perreault and Leigh’s Ir are patterned, in terms of the number of judges involved and the categories of answer defined. The outcome underlines the importance of...
متن کاملthe impact of portfolio assessment on iranian efl students essay writing: a process-oriented approach
this study was conducted to investigate the impact of portfolio assessment as a process-oriented assessment mechanism on iranian efl students’ english writing and its subskills of focus, elaboration, organization, conventions, and vocabulary. out of ninety juniors majoring in english literature and translation at the university of isfahan, sixty one of them who were at the same level of writing...
15 صفحه اولAutomated Coding of Open-Ended Survey Responses
Background: Manual coding of open-ended survey responses is time-consuming, expensive, and error-prone. However, this task can be recast as a multi-label text classification problem, to which natural language processing and machine learning techniques can be applied. Aim: The aim of this work is to assess the suitability of various machine learning techniques to the problem of automated coding ...
متن کاملAutomatic Multi-criteria Assessment of Open-Ended Questions: A Case Study in School Algebra
This paper deals with authoring assessments of complex competence involving open-ended questions. We present, PépiGen, a multi-criteria automatic assessor for school algebra, via a walkthrough of an example. PépiGen is based on our previous work on Pépite, an automatic cognitive diagnosis tool that capitalizes on educational research results. From that prototype, we derived patterns of diagnosi...
متن کاملAutomatic Assessment of Open Ended Questions with a Bleu-Inspired Algorithm and Shallow NLP
This paper compares the accuracy of several variations of the BLEU algorithm when applied to automatically evaluating student essays. The different configurations include closed-class word removal, stemming, two baseline wordsense disambiguation procedures, and translating the texts into a simple semantic representation. We also prove empirically that the accuracy is kept when the student answe...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Quality & Quantity
سال: 2006
ISSN: 0033-5177,1573-7845
DOI: 10.1007/s11135-005-1093-6